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Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )^| Responsive to communication(s) filed on 14 August 2008 . 
2a )□ This action is FINAL. 2b)£3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^3 Claim(s) 1-43 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) |EI Claim(s) 1-43 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) ^ The specification is objected to by the Examiner. 

10)^ The drawing(s) filed on 13 August 2001 is/are: a)^ accepted or b)^ objected to by the Examiner. 
Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 0 Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 

3.Q Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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Claim Status 

Claims 1-43 are pending 

Specification 

The disclosure is objected to because it contains an embedded hyperlink and/or other 
form of browser-executable code. Applicant is required to delete the embedded hyperlink and/or 
other form of browser-executable code. See MPEP § 608.01. 

Page 1 1 includes an embedded hyperlink. 

Claim Rejections - 35 USC §102 

The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

Claims 1-43 are rejected under 35 U.S.C. 102(e) as being anticipated by Beranek (US 
6,886,013), hereafter Beranek. 
Regarding claim 1, 17, 33, Beranek discloses: 

(a) at least one multimedia information input interface receiving said multimedia information; 
[Figs 1,2 and 3] 
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(b) a computer processor, coupled to said at least one multimedia information input interface, 
receiving said multimedia information therefrom, processing said multimedia information by 
performing object extraction processing to create multimedia object descriptions from said 
multimedia information, and processing said created multimedia object descriptions by object 
hierarchy processing to generate create multimedia object hierarchy descriptions indicative of an 
organization of said object descriptions, wherein at least one description record including said 
multimedia object descriptions and said multimedia object hierarchy descriptions is created for 
content embedded within said multimedia information; and 

[Fig 6, Fig 12, col 2, lines 48-55, col 3, lines 1-15, col 8, lines 36-55, col 9, line 7 - col 10, line 
65, col 11, line 40-50, col 12, lines 25-60, col 13, lines 40-60, Fig 12] 

(c) a data storage system, operatively coupled to said processor, for storing said at least one 
description record [Fig 3, 227]. 

Regarding claim 2, 18, 34, Beranek discloses wherein said multimedia information comprises 
image information, said multimedia object descriptions comprise image object descriptions, and 
said multimedia object hierarchy descriptions comprise image object hierarchy descriptions [Fig 
6]. 

Regarding claim 3, 19, Beranek discloses 

(a) image segmentation processing to segment each image in said image information into regions 
within said image; and 

(b) feature extraction processing to create one or more feature descriptions for one or more of 
said regions; whereby said created object descriptions comprise said one or more feature 
descriptions for one or more of said regions [col 9, lines 5-50]. 
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Regarding claim 4, 20, 35, Beranek discloses wherein said one or more feature descriptions are 
selected from the group consisting of text annotations, color, texture, shape, size, and position 
[col 9, lines 5-50]. 

Regarding claim 5, 21, 36, Beranek discloses wherein said object hierarchy processing comprises 
physical object hierarchy organization to generate create physical object hierarchy descriptions 
of said image object descriptions that are based on spatial characteristics of said objects, such 
that said image object hierarchy descriptions comprise physical descriptions [col 9, lines 5-50] 
Regarding claim 6, 22, 37, Beranek discloses wherein said object hierarchy processing further 
comprises logical object hierarchy organization to create logical object hierarchy descriptions of 
said image object descriptions that are based on semantic characteristics of said objects, such that 
said image object hierarchy descriptions comprise both physical and logical descriptions [col 9, 
line 5 through col 10, line 65]. 
Regarding claim 7, 23, Beranek discloses 

(a) image segmentation processing to segment each image in said image information into regions 
within said image; and 

(b) feature extraction processing to create object descriptions for one or more of said regions and 
wherein said physical hierarchy organization and said logical hierarchy organization create 
hierarchy descriptions of said object descriptions for said one or more of said regions [col 9, line 
5 through col 10, line 65]. 

Regarding claim 8, 24, 32, Beranek discloses an encoder receiving said image object hierarchy 
descriptions and said image object descriptions, and encoding said image object hierarchy 
descriptions and said image object descriptions into encoded description information, wherein 
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said data storage system is operative to store said encoded description information as said at least 
one description record [col 15, lines 10-30]. 

Regarding claim 9, 25, 38, Beranek discloses video information, said multimedia object 
descriptions comprise video object descriptions including both event descriptions and object 
descriptions, and said multimedia hierarchy descriptions comprise video object hierarchy 
descriptions including both event hierarchy descriptions and object hierarchy descriptions [col 
10, lines 25-40] 

Regarding claim 10, 26, Beranek discloses 

(a) temporal video segmentation processing to temporally segment said video information into 
one or more video events or groups of video events and create event descriptions for said video 
events, 

(b) video object extraction processing to segment said one or more video events or groups of 
video events into one or more regions, and to create object descriptions for said regions; and 

(c) feature extraction processing to generate create one or more event feature descriptions for 
said one or more video events or groups of video events, and one or more object feature 
descriptions for said one or more regions; 

wherein said created video object descriptions include said event feature descriptions and said 
object descriptions [col 10, lines 27-40] 

Regarding claim 1 1, 27, 39, Beranek discloses wherein said one or more event feature 
descriptions are selected from the group consisting of text annotations, shot transition, camera 
motion, time and key frame, and wherein said one or more obj ect feature descriptions are 
selected from the group consisting of color, texture, shape, size, position, motion, and time [col 
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9, lines 5-50]. 

Regarding claim 12, 13, 14, 15, 16, 28, 29, 30, 31, 40, 41, 42, 43, Beranek discloses wherein said 
object hierarchy processing comprises physical event hierarchy organization to generate create 
physical event hierarchy descriptions of said video object descriptions that are based on temporal 
characteristics of said video objects, such that said video hierarchy descriptions comprise 
temporal descriptions [col 12, lines 25-40]. 



Contact Information 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Etienne P. LeRoux whose telephone number is (571) 272-4022. 
The examiner can normally be reached on Monday through Friday, 8:00 am - 4:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Apu Mofiz can be reached on (571) 272-4080. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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/Etienne P LeRoux/ 

Primary Examiner, Art Unit 2161 



